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Abstract —In this paper, we present a novel framework that 
can he used to study the capacity and power scaling properties 
of linear multiple-input multiple-output (MIMO) dx d antenna 
amplify-and-forward (AF) relay networks. In particular, we 
model these networks as random dynamical systems (RDS) and 
calculate their d Lyapunov exponents. Our analysis can he 
applied to systems with any per-hop channel fading distrihution, 
although in this contrihution we focus on Rayleigh fading. Our 
main results are twofold: I) the total transmit power at the nth 
node will follow a deterministic trajectory throngh the network 
governed hy the network’s maximum Lyapunov exponent, 2) the 
capacity of the ith eigenchannel at the nth node will follow 
a deterministic trajectory through the network governed hy 
the network’s ith Lyapunov exponent. Before concluding, we 
concentrate on some applications of our results. In particular, 
we show how the Lyapunov exponents are intimately related 
to the rate at which the eigenchannel capacities diverge from 
each other, and how this relates to the amplification strategy and 
number of antennas at each relay. We also use them to determine 
the extra cost in power associated with each extra mnltiplexed 
data stream. 

Index Terms —Relay network, amplify-and-forward, AF, 
MIMO, capacity, affine, random dynamical system, RDS, Lya¬ 
punov exponent, scaling, finite antenna. 

I. Introduction 

C ONSIDER a multiple-input multiple-output (MIMO) link 
with ds source antennas and djj destination antennas. 
It is well known that, under some basic assumptions (i.e., 
independent channel fading between each antenna pair), the 
capacity will almost surely scale linearly with min{ds, du}, 

Q. 

Now, consider an n-hop MIMO link, aided by n — 1 relay 
node^ where each relay node is equipped with d transmit and 
receive antennas. Furthermore, assume that signals received 
at the ith relay node propagate only as far as the {i + l)th 
node (Fig. [^l. The end-to-end capacity, Cn, of such links has 
been studied in many works: see, e.g., 0-0 for amplify-and- 
forward (AF) studies, and ©-HD for decode-and-forward 
(DF) studies. It is known, that the capacity of such net¬ 
works is achieved by employing DF relaying. However, when 
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* The deployment of relays is interesting because it can increase the diversity 
gain ID, and extend the coverage area of the network j^. 


larger numbers of nodes are deployed, DF-based protocols 
may result in prohibitive latency/complexity because of the 
decoding process that takes place at each relay. AF protocols 
become interesting at this point since they can be employed 
to yield low complexity and/or low latency solutions. Under 
certain scenarios, they also have the potential to offer greater 
diversity when compared to DF schemes Q- 

The analysis of DF networks in the general multihop setting 
is made easier by the fact that a local view can often be 
taken - i.e., the transmission is “reset” at each relay node, 
and thus sequential hops can, to a certain extent, be treated 
independently. This is not the case for AF networks, which 
must be observed globally in the general case since the end- 
to-end transmission is affected by a composition of mappings, 
one for each hop. Consequently, although AF relay networks 
may exhibit potential in multihop applications, relatively little 
is known about how these systems scale. 

A number of results exist pertaining to AF relay networks 
that scale in size. For a linear n-hop network, it was shown 
in 0, © that lim„_>oo [lirnd^-^oo [cn/dc]/n] exists al¬ 
most surely and is strictly positive, provided d/d^ scales 
at least linearly with n (i.e., d/do — U (n)). This work 
also considered the aforementioned limit for other forwarding 
strategies; namely, DF, compress-and-forward, and quantize- 
and-forward. In |T3), the asymptotic (in matrix dimension) 
eigenvalue distribution of the channel’s covariance matrix 
for linear n-hop MIMO channels with noiseless relays was 
established. Using free probability theory and, again, under 
the premise that negligible noise was received at the relays, 
it was shown in © that when linear precoding was applied 
at each relay, c„ would converge almost surely to a limit as 
d grows large. The singular vectors of the optimal precoding 
matrices for such a network when noise was negligible at the 
relays was also established in ©. When noise was present at 
the relays, ergodic capacity and average bit error rate results 
were established in ||7) for multihop AF MIMO networks when 
arbitrary signaling occurs at the source node and, again, d 
grows without bound. Meanwhile, in p4| , c„ was assessed 
for general n-hop AF networks in terms of the limiting (in d) 
eigenvalue distribution of products of random matrices when 
noise was not negligible at the relay nodes. Related work on 
the diversity-multiplexing tradeoff, p3| , for various MIMO 
multihop relaying strategies can be found in ©, p6)-|fT8t. 

Other attempts to determine the behavior of AF networks 
as they scale (not necessarily in the number of hops) can be 
found in ID-ID- In more detail, ID considers a network 
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Fig. 1. An illustration of a linear relay network. The zth hop’s channel is 
described by the channel matrix H^. The noise received at the jth node is 
described by the vector Vj. Nodes 0 and n are the source and destination, 
respectively. 


of m source-destination nodes communicating through a set 
of k relays in a two-hop fashion. It is shown that, provided k 
grows fast enough with m, in the large m limit the network 
will “crystallize” into a set of nonfading source-destination 
links with strictly positive capacity. In pO) a single-antenna 
source-destination pair aided by layered relays is studied. It 
is shown that such networks will approach the cut-set bound 
as the received power at each relay increases. An m antenna 
source-destination pair assisted by k single-antenna two-hop 


relays is studied in |211, where it is shown that for fixed m, 


the capacity of the network will obey c = m/2\ogk -1-0(1) 
as k ^ oo. 

To the best of the authors’ knowledge, all attempts to study 
the statistical behavior of the end-to-end capacity for n-hop 
(n > 2) AF MIMO networks have leveraged a viewpoint in 
which the number of antennas at each node grows larg^ To 
achieve this, results from random matrix theory p3j have 
commonly been employed; e.g., |[24)-p6|, which describe 
the asymptotic/limiting spectral properties of large random 
matrices. The limitations of such approaches are that statistical 
spectrum behavior is established only at a macroscopic scale. 
The term macroscopic scale is commonly used in random 
matrix theory to describe macroscopic/'global’ observables 
such as the empirical eigenvalue distribution (see Ez) for more 
details). In this contribution, we establish statistical laws for 
each of the spectra individually. Crucially, this allows us to 
determine capacity scaling laws for each of the subchannels 
of a network when the number of antennas at each node is 
finite. This is done by employing the formalism of random 
dynamical system^ (RDS), Such systems have also 

been used to study econometrics |29|, biological systems H), 
ED, chemical reactio ns ||3D , and the propagation of particles 
through fluidic media ]32[. For relevant information on RDSs, 


the reader may refer to section IV 


Going into more detail, the Lyapunov exponents p8| of 
RDSs are known to characterize the exponential growth/decay 
rates of the spectrum of finite dimensional random matrix 
products 


|33|, 


. In this contribution, we use these 
Lyapunov exponents to study the spectral properties of the n- 


^For n = 2 hops, results have been obtained for finite antenna systems 
(see, e.g., (22) ). 

^The RDS formalism can be applied to any relay system that can be de¬ 
scribed as a product of random matrices. This encompasses all AF strategies. 
Because of the ‘resetting’ nature of digital relaying protocols (e.g., DF), it 
is unclear whether the RDS formalism can be employed to study the scaling 
properties of such networks. 


hop AF MIMO networlQ The main result of our paper is that 
the Lyapunov exponents of the network, which are obtained 
by studying the network as an RDS, can be used to evaluate 
the exponential growth/decay of the nth node transmit power 
and nth node end-to-end eigenchannel capacity when each of 
the nodes in the network has a finite number of antennas. 

A. Notation and Definitions 

We use N, K and C to denote the natural, real and complex 
numbers. We use = to denote equality in distribution, := 
to denote equality by definition, logarithms are always to 
the base e and log^(x) := max{0,logx}. 0 is used to 
denote the column vector of zeros, where the dimension of 
0 will be implied from the context. Matrices are always 
represented using uppercase boldface notation, vectors are 
always represented using uppercase non-boldface notation, 
and scalars are always represented using lowercase notation. 
8i{K] is used to denote the *th ordered eigenvalue of the 
matrix A, where £i{A) > £j{A) implies i < j. At is used 
to denote the conjugate transpose of the matrix A. Matrix 
products are defined in the following way: 

n 

Ha, :=A„-..A„ (1) 

and when j = 1 we sometimes use the definition 

n 

7r„(A):=nA,. (2) 

i=l 

The standard 2-norm of a matrix A is denoted by ||A|j, and 
its Frobenius norm is denoted by ||A||p,. The Landau notation 
f{x) = o{g{x)) is used to imply f{x)/g{x) = 0. 

Also, we use the following notation: 


f{n) = 0 {g{n))^3ki,n>0 s.t. ki\g{n)\ > |/(n)|,Vn > n 
f{n) = n{g{n))^3k2,n>0 s.t. k 2 \g{n)\ < |/(n)|,Vn > n 
f{n) = 0 {g{n)) if /(n) = O {g{n)) and /(n) = n {g{n )); 


and, similar to the notation proposed in p7| , for a strictly 
positive random variable f(n) depending on n, and some 
h{n) = |o(n)|. 


f{n) = Ov[g{n)) 
f{n) = np {g{n)) 


limP 


limP 

n—^oo 


/(n) < 5 (n)e'‘("') 
f{n) > 5 (n)e-''(" 


= 1 . 


= 1 


/(n) = 0p {g{n)) if f{n) = Or {g{n)) and f{n) = f2p {g{n)). 


B. Paper Layout 

Section |n] introduces the system model. Sectioning clarifies 
the key results obtained in this paper. Section |IV| introduces 
the mathematical preliminaries and new RDS results that will 
be utilized throughout this paper. In section |V] we calculate 
the Lyapunov exponents, and show that they can be used 
to characterize the network’s transmit power and end-to-end 
eigenchannel capacity. Section |Vl] establishes applications of 

^Lyapunov exponents were used in |35| , (36) , where sum-capacity scaling 
laws were established for the non-ergodic Wyner cellular model as the number 
of cells grew large. In more detail, the upper Lyapunov exponent is used in 
|35| , |36| by considering the Thouless formula, which relates the determinant 
of a large random matrix to a product of fixed size matrices. 
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the results that are obtained in section|V] Section [Vn| provides 
numerical illustrations of the theory that has been developed. 
Finally, section VIII concludes the paper. 

II. System Model 


Let us present the signaling model used in this paper. 
Consider an n-hop AF relay network, as depicted in Fig. 
We assume that each node has d > 1 transmit and receive 
antennas. Independent frequency-flat Rayleigh fading p8) is 
assumed to take place between each node pair. Thus, the 
channel for the ith hop can be described by a d x d random 
matrix. Hi, whose elements are circularly symmetric complex 
Gaussian pS) with total variance /i^; i.e., the (a, &)th element 
of Hi is given by hab,i 

At each node (apart from the zeroth node; i.e., the source) 
we assume noise is introduced into the system. We use 
Vj G to denote the vector of noise terms introduced at the 
jth relay. The elements of Vj correspond to the noise samples 
received at each antenna of node j and are independent 
complex Gaussian random variables with total variance uq. 

An information vector 


-^0 — [to, 1 , 


7 


T 


(3) 


is constructed at the source (node 0). We assume each element 
of Xq has a mean of zero and average power given by 
]E[|To,ip] = Po/d. The ith element of Xq is then transmitted 
from the ith antenna of node 0. 

We assume the jth relay node receives the transmission only 
from the (j —l)th node in one time slot. This relay then applies 
a scalar gain, aj, to the received signal on each of its antennas 
and transmits in the next time slot. Thus, the relays operate in a 
half-duplex manner. The gain for the jth relay is either a fixed- 
gain parameter, depending only upon the average statistics of 
the channel matrix of the previous hop, given by 


/.= 


Pj 


Pj-idpj + dno 


(4) 


or a variable-gain parameter given by |39 eq. (7)] 


Vi = 


Pi 




drin 


(5) 


The term pj is selected by the relay, and represents the 
average transmit power at node j. Also, we assume that 
lim„_>oo(l/«) log(pn/Po) < oo. This assumption implies 
that the average transmit power does not grow at a super¬ 
exponential rate. Similar assumptions have also been made in 
0. It is important to note that we are implicitly assuming 
the relays have access to statistical channel state information 
in the form of pi for fixed-gain and ||H^ |j^ for variable-gain. 
The precise mechanism by which these are obtained is beyond 
the scope of this paper. Needless to say, using tools such as 
received signal strength indicators, it is possible for these to 
be gleaned from a channel output without having to perform 
decoding operations at each relay. 

The information bearing content of the signal (herein re¬ 
ferred to as the information component) at the nth node is 


given by 


-ria) _ 


r(“) 


antinin-l 


i=i 


( 6 ) 

(7) 


where a G {f,v} dependent upon whether fixed-gain or 
variable-gain is being implemented. Similarly, the total trans¬ 
mitted signal at the nth node is given by 


( 


+ Vj 


n n 


i=l j=i+l 


K 


(c) 


( 8 ) 

(9) 


where iHjl4, := Vn and denotes the 

accumulated noise at node n. Owing to our choice of gain, 
and by the definition of the source transmit vector for all 
i G N U {0} the ith node is subject to the following average 
power constraint; 


E 




< Pi¬ 


rn 


The channel input, 0, can be re-expressed as the first d 
entries of the following matrix product: 



n 




Xo 

\ ’ 


( 11 ) 


where 






( 12 ) 


and Zi = Vify/nQ. This matrix formulation will help us to 
establish power scaling laws for the network. 

Finally, we give a definition for the capacity of the network 
described above. 

Definition 1: The information capacity of the n-hop net¬ 
work with amplification strategy a and average power con¬ 
straint Pi for the ith node is 

= max / (Ao; , (13) 

Pxg(X) \ / 


where Pxg denotes the probability density function of the 
source vector, Xq, and I denotes the mutual 

information between Aq and Ai“^/a„. 

Fact 1: From Q, the end-to-end channel capacity of the 
network, ([T3), is given by 

0 ^“) = logdet (id + (14) 

d 

= ^ (nats/channel use), (15) 

i=l 


where 


n j ... HiHl... Hi, 

/ n n 

Ra.I = no(ld + .. • HiHl ... Hi, 


1=2 i=l 


(16) 

( 17 ) 
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are covariance matrices; and = 

log ^1 + is the capacity of the ith 

eigenchannel at the nth node. 

III. Key Results 

One of the key insights that we provide in this paper is that 
n-hop AF MIMO systems can be studied from the viewpoint 
of RDS. To the best of our knowledge, this is the first time such 
an approach has been taken in the literature. This viewpoint 
then leads us to obtain the following results; 

• In Lemma and Theorem we show that the d x d 
antenna MIMO AF network has associated with it an 
ordered set {Aqh.i, ■ • ■ , AaH.d} of Lyapunov exponents 
satisfying 

AqH.I > • • • > XaH,d, 

where the a € {f,v} term in the subscript denotes the 
amplification strategy that is being implemented (i.e., / 
for fixed-gain or v for variable-gain). From this ordered 
set, two other sets of exponents are established. The 
first of these sets is constructed from elements of the 
form Aq^j = max{AQH.uO}> is associated with the 
instantaneous total transmit signal at the nth node. The 
second of these sets is constructed from elements of the 
form A^“^ = min{2AQ,H.i, 0}, and is associated with the 
end-to-end SNR of the network’s d eigenchannels. 

• In Lemma S we show that the instantaneous transmit 
power, ||A1„ Ip, at the nth node obeys the relationship 

= , (18) 

where, /(n) = Qr{g{n)) implies that /(n) is equal 
to gin), to first order in the exponent eq. (3.26)]. 
This is defined more rigorously in the notation subsection 
(subsection E3- 

• In Theorem]^ we show that the SNR and capacity of the 
ith eigenchannel at the nth node, 

satisfy the relations 

and = 0p , (19) 

where Rj“^ and R^^„ are given in CD and 
respectively. 

On top of our main results, we also establish the following 
notable secondary results: 

• In Lemma we show that to ensure the instantaneous 
transmit power almost surely displays no exponential 
growth, and that the end-to-end capacity of the dominant 
eigenchannel almost surely displays no exponential decay 
(i.e., from ( |T8] l and (|T^, Aqh.i = 0), the average trans¬ 
mit power must grow exponentially with n. Furthermore, 
this rate of average power growth can be reduced by: 

1) implementing variable-gain instead of fixed-gain, 

2) increasing the number of antennas at each node. 

• In ( [56] l, we show that the exponential rate at which the 
capacities of the ith and jth (i < j) eigenchannels diverge 


away from each other is given by n ^A^^]© — A^“j ^. 
When the ith and jth eigenchannel capacities are either 
both decaying or both not decaying, this divergence rate 
is shown to be independent of whether fixed-gain or 
variable-gain relaying is being performed. Furthermore, 
from Lemmaj^and Corollarywith * = 1, to ensure that 
this rate is asymptotically bounded away from infinity (so 
that multiplexing j streams is asymptotically viable) we 
must either; 

1) ensure that Aqh.j > 0^ 

2) ensure that the number of antennas at each node 
grows like d = n(n). This result complements those 
presented in Q, Q. 

• In Remark we assign a transmit power cost to the 
nth node for each extra data stream that is multiplexed 
over the network. In particular, if i data streams are being 
multiplexed, then, to multiplex one extra stream, we must 
increase the nth relay’s instantaneous transmit power by 
a factor of exp(n/(c? — i)). 

On the way to proving the above mentioned results, we 
also obtain the following RDS results, which we believe are 
of independent interest. 

• For i e N, let S £dxd ^ £d random 

matrices and vectors, respectively, with Elog^ l|Ai|| < 

a.s. 

00 , Elog’*' ||i?i|| < oo, and Ri ^ 0. Suppose that there 
exists aj,i3j G M such that Ai is equal in distribution to 
ajAj and Ri is equal in distribution to PjRj. In Lemma 
[T] we show that the Lyapunov exponents of an affine RDS 
taking the form 

Xn = An An-1 + Rn, ( 20 ) 

are strictly positive, and, consequently, are identical to 
those of 


■ ■ 


r A 

■^n 

Rn 


■ Ai 

Ri ' 


■ Ao ■ 

1 


QT 

1 


qT 

1 


1 


( 21 ) 

• In Theorem [2 we show that the Lyapunov exponents of 
( |2 T] i are given by the non-negative Lyapunov exponents 
of 

7rn(A) := A„ • • • Ai. (22) 

Less formally, our RDS results provide us with a framework 
for determining all of the Lyapunov exponents of d dimen¬ 
sional affine RDSs, which will be crucial to our information 
theoretic analysis. 

IV. Random Dynamical Systems 

In this section, we introduce the RDS results that will be 
relied upon heavily throughout this paper. The first subsection 
is devoted to presenting preexisting RDS theory, while the 
second subsection presents a new result which will be used to 
calculate the Lyapunov exponents of affine systems. 

A. Preliminary RDS Results 

The study of dynamical systems is concerned with tracking 
the trajectory of a position (particle/state/point) through a state 
space. In the discrete case, this position is calculated through 
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the repeated action of a deterministic map. Informally, an RDS 
occurs when this map is non-deterministic and drawn from a 
sample space according to some hxed probability distribution. 
Such systems are are often used to study econometrics | [29) , 
biological systems pO) , [31] , chemical reactions pT| , and the 
propagation of particles through fluidic media p^ . The formal 
and rather intricate dehnition of an RDS can be found in pS) . 

In this contribution, we consider an RDS to be the action 
of a product of dxd {d gN) complex random matrices on an 
appropriately dimensioned vector (the initial state Xq G C'^). 
The state of the RDS at time n (Xn G C^) can then be written 
as either 


Xji — Xji • • • A^Aq 

, (23) 

or 


Xn — Ai • • • A„Ao 

(24) 

where, in general, we assume that Ai,... 

, An are independent 

and identically distributed (i.i.d.) up to 

an arbitrary positive 


scaling factor. Mathematically, this means that 3 > 0 such 

that A-x — for all i. Ec|s. and are referred to 

as forward and backward RDSs, respectively, and take their 
names from the forward pS] Def’n. 1.1.1] and backward p^ 
Rem. 1.1.10] cocycle properties that their random mappings 
satisfy. It is interesting and important to note that, unlike p^ , 
p4l i is somewhat unnatural, in the sense that it is anticausal; 
however, all of the RDS properties that are to be described for 
P^ will apply to p4l i as well, p8) . 

Suppose we wish to study the asymptotic behavior of 

lk„(A)|| (25) 

as n —oo. A traditional approach is to exponentiate the 
logarithm of the norm; i.e., write p5] l as 

||7r„(A)|| = (26) 


The Lyapunov exponent can be used to describe the expo¬ 
nential growth rate of the norm of a trajectory through its state 
space, where the initial state of the trajectory is given by X. 

Remark 1: In the dehnitions of the Lyapunov index and 
exponent ( p7] i and p8] l, respectively), if the system is linear 
then the limsup can be replaced with a limit, p8) . 

Fact 2: From |28 pp. 114 — 115, Theorem 

E[log''’ ||Ai|j] < oo and —oo < 


From 
3.3.3], assuming 


lim^^_^oo 1 /^Z ]"=1 l«i| < OO (where Ai = a*Ai), 
the Lyapunov exponent has the following properties | [28| : 

1) A(A,A) G K U {— c»} V X G C'^, where A(A, 0) := 
—oo; 

2) The number, p, of distinct values, Ai, that A(A, A) can 
take on for X G C'^\{0} is at most d, and we have 


—00 < Ap < • • • < Ai < oo. 


3) The sets 


V^ := {X : A(A,A) < AJ 


(29) 


are linear subspaces, form a hltration 

{0} =: Vp+i C Vp C • • • C Vi = (30) 

(where all inclusions are proper), and 


A, = A(A,A)4^Ae VAV.+i, i = (31) 

The integer m(i) := dim — dim)2j_i is the multiplic¬ 
ity of A;. 

4) The limiting behavior for the ordered singular values of 
the matrix product 7r„(A) satishes 

^ logfi {7r„(A)7r„(A)^} A^. (32) 

Consequently, the random variable Si {7r„(A)7r„(A)^} 
satishes 


and investigate the behavior of the exponent, specihcally, the 
term A log ||7r„(A)|| as n grows large. In this manner, the 
exponential growth/decay rate of the system can be observed. 
If {Aj} was a set of scalars (i.e., d = 1), the law of large 
numbers could be employed to evaluate the limiting behavior 
of A log ||7rji(A)||; however, this is not the case for general d. 

The question of whether A log ||7r„(A)|| tends to a 
limit does not have a clear answer in most cases. 
Under the condition that E[log~'' ||Ai||] < oo and 

lim„_>oo l/^Sr=i ^ however, the theorem of 

Furstenberg and Kesten | [28) guarantees that the limit exists. 
We then obtain the Lyapunov index: 

Definition 2: The Lyapunov index is given by 

t(A) := limsup - log ||7r„(A)|| . (27) 

n—^oo ^ 


The Lyapunov index can be used to describe the exponential 
growth rate of ||7r„(A)||. By evaluating the Lyapunov index 
at a specihc initial position within the state space, we then 
obtain the Lyapunov exponent: 

Definition 3: The Lyapunov exponent is given by 


A(A, A) := limsup - log ||7r„(A)A|| . 

n—>-oo ^ 


S, {7r„(A)7r„(A)t} = 0p . (33) 

In what follows, we will often drop the functional notation 
A (A, A) and simply write Aa.^ to refer to the ith ordered 
Lyapunov exponent of the system corresponding to 7r„ (A). 
When it is clear, we may also omit the subscript A as we did 
in Fact |2] 


B. On the Lyapunov Exponents of Ajfine RDS 

Throughout this paper, we will often be concerned with the 
Lyapunov exponents of affine systems of the form 


A„ — A„A„_i + Rn, (34) 


where A„ G is a random matrix and i?„ S is a 

d 

random vector that satishes i?„ = — i?„ and f 0. The 
following theorem will be used in the calculation of these 
exponents. 

Theorem 1: For j S N, consider the product of random 
matrices 7r„(M), where 


M,; = 


Ai R, 

ai 




(28) 


(35) 
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Elog'*' ||Ai|| < oo, Elog'*’ ||iii|| < oo, and Elog'*’ |ai| < oo. 
Under the assumption that 7r„(A) has d distinct Lyapunov 
exponents, V i 3 Ag G C‘^ such that 

a(^M,[aJ = max{AA,i,Aa,i}. (36) 

Proof: See Appendix [A| ■ 

1) Applications and/or Implications of Theorem We 

will now show that Theorem [T] can be used to calculate the 
Lyapunov exponents of ( [34| ). To do this, the affine structure of 
( |3^ will be captured by converting it into a linear (non-affine) 
(d + 1) X (d + 1) system of the following form; 


■ ■ 


■^n 

Rn 


■ Ai 

Ri ' 


■ Ag ■ 

1 


0^ 

1 


qT 

1 


1 


One may naively assume that the Lyapunov exponents of ( |34| ) 
are trivially identical to those of ( [37] i. However, for this to be 
true, we must have lim„_).oo ^ log || A„|| > 0, because, clearly. 


lim — log 

n—^oo 77 , 


[A„ 1]^ 


> 0 . 


(38) 


We now provide the following lemma, which tells us that the 
Lyapunov exponents of ( |34| l are indeed strictly non-negative, 
and that, consequently, the Lyapunov exponents of ( |34l l and 
•HU are identical. 

Lemma 1: For A„ given by we have 
lim - log ||A„|| > 0. 

n—^oo Ji 


Consequently, the Lyapunov exponents of the affine system, 
and those of the linear-to-affine converted system, ([33, 
are identical. 

Proof: See Appendix [B] ■ 

From Fact (below), it can be seen that the Lyapunov 
exponents of pT] ! (and consequently (0) must belong to 
{max{AA,i, 0} : z = 1,... ,(i} . 

Fact 3: pd] Theorem 5], consider the product of random 
matrices 7r„(M), where 


M, = 


A, 

qT 


Ri 

ai 




(39) 


Elog'*' ||Ai|| < 00 , Elog"*" ||i?i|| < oo, and Elog"*" |ai| < oo. 
Then the Lyapunov exponents of 7r„ (M) are given by {Aa,* : 
z = 1,..., n} U Aa 1 . Furthermore, the Lyapunov exponents of 
7r„(M) are independent of the statistics of Rt. 

Notice that Fact tells us nothing about how initial states 
of the form [Xj l]^ (c.f. ([J7|)), will affect the Lyapunov 
analysis. Consequently, we require a theorem that deals with 
such initial states. This provides the rationale behind Theo¬ 
rem [2 It is important to mention that TheoremJT] also plays 
an important role within the proof of Theorem |2] 


V. Capacity and Power Scaling 

In this section, we will use the network’s Lyapunov ex¬ 
ponents to establish the scaling behavior of the end-to-end 
capacity and nth node transmit power. The following theorem 
relates the Lyapunov exponents to the network’s end-to-end 
capacity. 


Theorem 2: Let Aq-h,* be the Lyapunov exponent 
of (which is dehned in (|^) and 


E d 

i=lC. 


(° 

Cn 


= i n ,2 


(nats/channel use) be the capacity of the 
n-hop AF network with amplihcation strategy a (see 
Definition 1 and Fact iL where is the capacity of the zth 
eigenchaniS at the nth node. Then the following statements 


hold. 

A. 


^logf. (: 


T) (^) Ty 

^X,n^N',n 


liniyj—^oo 

min{0,2AaH.J =: 

the zth eigenchannel obeys 


a 1“^ Hence, the SNR of 

7)* 


„(“) 


B. 


lim„ 


(40) 


“=■ min{0,2AaH,i} =: A, 
Thus, the capacity of the zth eigenchannel obeys 


(a) 

'7,2 ■ 


= e. 




(41) 


Proof: See Appendix [C] ■ 

The next lemma will evaluate the Lyapunov exponent Ach.a 
and establish how it relates to the Lyapunov exponents of 
and the average transmit power at the nth node. This lemma 
will in turn allow us to establish a trade off between capacity 
decay and power growth across the network. It will also have 
implications on gain design. 

Lemma 2: With given by (|^, given by (|^, and 
the average transmit power at the nth node given by p„, the 
following statements hold. 

The zth Lyapunov exponent of is given by 

AaH,* = + i’{d - z + 1)), z = 1,- • d; 

(42) 


A. 


where ip (•) is the digamma function |42 eq. (6.3.1)] 
and 

1 " 

L{a^p)-= lim - V log(a,^^i). (43) 

n.—400 n < ^ 


The zth Lyapunov exponent of is given by 

^Qi •= ^ (Q> [^0 V^]) = max{0, AaH,*} • 

(44) 

Hence, the information power and total transmit 
power obey 


t(“) 

a7) 


B. For hxed-gain, we have 


0P 

0P 


)■ 


(45) 

(46) 


1 T) 

lim — log — > max{2AfH i + logd — ip{d), 0} ; 

riA-oo n Pq 

(47) 

for variable-gain, we have 


1 


lim — log — > max {2 A„h i + 'P{d'^) 

n->oo n po 

-ip{d)-logd, 0}; (48) 

where equality is maintained only when ng = 0. 
Proof: See Appendix [P] ■ 
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A. A Brief Discussion of Theorem and Lemma 

From the first statement of Lemma by ensuring < 

• • • < Aqh.i = 0, we can avoid exponential growth in the 
instantaneous transmit power. However, in this setup Theorem 

tells us that all but the first eigenchannel will display 
an exponentially decaying capacity. Conversely, by ensuring 
AaH.l > ■ ■ • > AqH.ij > 0 > AcH.g+l > ' ’ ’ > AcH.d, We 
can stop the end-to-end capacity of the upper q eigenchan- 
nels from almost surely decaying exponentially. However, in 
this scenario, we must allow for exponential growth in the 
instantaneous power across the network. Thus, there is a clear 
tradeoff to be had between multiplexing multiple data streams 
across the network, and growth in the instantaneous transmit 
power at the nth node. 

Focusing on the second statement of Lemma it can be 
seen that the terms logd — 4’{d) and — ipid) — logd 

in ( |47l i and ( |48] l, respectively, are strictly non-negative. Thus, 
this statement tells us that, asymptotically, the average trans¬ 
mit power must grow at a greater exponential rate than 
the instantaneous power. Crucially, we find that exponential 
growth in can be allowed for whilst avoiding (with high 
probability) exponential instantaneous power growth at the 
relays. Said in a different way, as the network scales in 
size, the density function of the transmit power at the nth 
node becomes increasingly heavy tailed. Whilst most of the 
distribution’s mass will be concentrated at the point governed 
by the Lyapunov exponent (cf. (|46ll), the distribution’s heavy 
tail will push the average up exponentially. Combining this 
observation with Theorem it can be seen that ensuring 
the first eigenchannel displays a non exponentially decaying 
capacity implies that the average transmit power will grow 
exponentially. 

It can also be seen that, because logd — '^{d) > f>{d^) — 
'0(d) — logd, the lower bound on the exponential growth rate 
of the average transmit power for variable-gain is strictly less 
than that for fixed-gain, which suggests that the variable-gain 
network can sustain an approximately constant instantaneous 
power trend with a reduced growth in the average transmit 
power. Furthermore, as the number of antennas grows large, 
both bounds in Lemma converge towards the Lyapunpov 
exponents. Thus, ergodic behavior is induced as d grows large. 

In summary. Theorem and the first statement of Lemma 
1^ expose a fundamental trade off between capacity decay 
and instantaneous transmit power growth across the network. 
The second statement of Lemma has important implications 
on gain design for scaled networks. In particular, it implies 
that the average transmit power at each node should grow 
exponentially with the network if an approximately constant 
instantaneous power trend is to be maintained. These impli¬ 
cations contrast with the system model proposed in 
where the capacity was assessed under strictly linear scaling 
of pn- For the finite antenna system, we see that if linear 
scaling of Pn occurs, lim„_>.oo log(p„/po)/’^ = 0 and (from 
Lemma 1^ Aqh.i < 0. As has been seen in the Theorem]^ 
AaH.i < 0 will have serious implications on the network’s 
end-to-end capacity. As an extra note, it can be seen that our 
result implicitly applies to a network whose length grows with 


the number of hops in the network (the extended regime); i.e., 
the distance between each node is fixed. For future work, it 
may be interesting to consider capacity and power scaling 
properties for networks when the end-to-end length of the 
network is fixed, and the distance between each of the nodes 
decreases with the number of hops (the dense regime), see 
| |43| . However, this is beyond the scope of this manuscript. 

Finally, the authors would like to point out that it is unclear 
whether our RDS and corresponding Lyapunov analysis can 
be applied to study other forwarding schemes (e.g., DF). For 
the AF scenario, the analysis relies on the ability to make a 
correspondence between the network and products of random 
matrices. Moving to other (non AF) forwarding scenarios, 
the relay network will be described by a composition of 
random nonlinear mappings. In general, when determining the 
Lyapunov exponents of a system, the multiplicative ergodic 
theorem | |28) (MET) is referred to. This theorem is a linear 
result, and when one refers to the MET for a nonlinear system 
they are implicitly referring to the application of this theorem 
to the linearized version of the nonlinear system. Thus, we 
have two open questions about applying our approach to other 
forwarding schemes: 

1) Does the linearization of a system describing forwarding 
such as DE make sense from a practical view point? 

2) If the answer to 1) is yes, can analogous capacity and 
power results be obtained for such schemes? 

VI. Applications of Theorem|2]and Lemma|2] 

In this section, we will study some applications of Theo¬ 
rem 1^ and Lemma In particular, we will study the rates 
at which the eigenchannel capacities diverge away from each 
other, and how this relates to: 

• the amplification strategy and number of antennas at each 
node, 

• the growth in the instantaneous transmit power. 

To discuss the above mentioned points, we will require the 
following preliminary definitions and lemmas. 


A. Preliminary Definitions and Lemmas 

Definition 4: The (f, j)th normalized channel capacity, i < 
j, is defined to be 






''i,n 
„(“)' 


(49) 


Clearly, if 


1, the channel will be well suited 


(a) 

-l.n 


for multiplexing j data streams, |38j, |44 , provided 
sufficiently large; otherwise, it wilTnot. 

Definition 5: Eor both fixed-gain and variable-gain, the 
(f, j)th Lyapunov difference, i < j, is defined to be 


j,(“)_\(“) _ 


(50) 


The following two lemmas are used to bound and will 
be employed in the ensuing analysis. 

Lemma 3: The (i,j)th Lyapunov difference is bounded as 
follows: 

< 2 (Aqht — AqHj) =: 0IJ) 


o<0: 




(51) 



2Aq,h,j - Consequently, in this scenario 


where lower equality is maintained if and only if Aqh.i > 
Aqh j > 0, upper equality is maintained if and only if Ao,h j < 
Aqh,* < 0, and = —2AQ,H,j otherwise. Furthermore, the 
upper bound is indepedent of whether hxed-gain or variable- 
gain is being implemented. 

Proof: See Appendix]^ ■ 

Finally, we will also exploit the following lemma later in 
this section. 

Lemma 4: For i < j, we have 


1 




= X! ^ = {'Hd-t - y-d-j), 


where Hi is the ith harmonic series dehned to be 

1 


^* = E 7 - 


i=i 


J 


(52) 


(53) 


Furthermore, by considering the hrst and last summands in 
b, we can trivially construct the following bound: 


—1-^ < (pi,j < -j -. (54) 

a — i a — j + 1 

Proof: This follows immediately from ( |4^ , Lemma 
and applying the telescope property of the digamma function 
eq. (6.3.5)]: 

1 


'0(a: + 1) = ip{x) + 


(55) 


B. Growth o /and HA'np 

We will now apply Theorem and Lemma to study 
(Definition!^ and Considering Theoremj^first, from 

Definitions g] and |5] we have 


lim — 

n—>-oo 77 , 


log!^, 


(«) 




(56) 


We will now use ( |56| ) (in conjunction with Lemma to study 
the following four problems: 

1) The dependence of the growth in on the amplifi¬ 
cation strategy. 

2) The dependence of the growth in v[°‘jrL ™ '^he number 
of antennas at each node 

3) The behavior of the network when either = 0 or 

ActH.i = 0; i-C-, when either display 

no exponential growth, respectively. 

4) The growth in n (^■®'’ which adjacent 

eigenchannel capacities diverge away from each other), 
and the cost (in terms of instantaneous transmit power) 
associated with each extra multiplexed data stream. 

1) Growth of Vi ond the Forwarding Strategy : Let us 
first establish how the amplification strategy affects the growth 
of i^i‘fn- ™ immediate consequence of Lemma 3 it can be 
seen that when AqHj < AqH/j < 0 the exponentialgrowth of 

i n "'ill I’® independent of the amplification strategy that has 
been implemented. The same holds true when 0 < Aqh.j < 
Aqh.i, since we will have = 0. For AqHj- < 0 < Aqh.i, 


we will have — ~ 
jy^j n IS given by 

= nr ^ (57) 

for fixed-gain, and 

4 t = Op 

(58) 

for variable-gain, where the second equalities of ( |57] i and ( |58| ) 
follow from Lemma(see Appendix]^. Notice that, because 
logd^ > V' 

g-lim„_,oo i log |^-|-logd-!/'(d-j-|-l) 

> g- i log |^-log(d)+i/i(d^)-V>(d-j-|-l) (5^) 


2 ) Growth of Vij^n ond the Number of Antennas: We will 
now establish how the number of antennas at each node will 

fa) 

affect the growth rate of j „■ In particular, we will determine 
how (the term in the exponent of ( [56] l) scales with n, and 
how the number of antennas relates to this. More specifically, 
in what follows (Lemma and Corollary we will determine 
conditions that give the following: 


lim 

n—¥oo 


0 < lim 

n—foo 


i(a) 


lim 

n—foo 



= 0 



< k < oo 


^^:3 


= CX) 

<;4- 


= 0 (i.e.. 

AaH,i 

> A 


= o{l/n), 

= 0 i^/n), 


(60) 

(61) 

(62) 


Of course, if = 0 (i.e., A^h,* > Aqh,j > 0) is 
obtained trivially. We are therefore only interested in studying 
the behavior of when either 0 > Aqh.i > Aq-h.^ or 

AaH.i > 0 > AdHj - We treat 0 > AaH,i > Aoh.j in Lemma|^ 
and consider A^ H.i > 0 > AqHj in its corollary. 

Lemma 5: When 0 > A^Hd > A^hj, for both fixed-gain 
and variable-gain, to leading order about d = oo (i.e., i < j 


fixed and d 


oo) is given by 

(j - *) 






oil 


(63) 


Consequently, 


1) the conditions that give (60l are 1/d = o(l/n), 

2) the conditions that give (61l are l/d = 0(l/n) 

3) the conditions that give (62i are d = o{n). 


Proof: Eq. ( |6^ is obtained by performing a Taylor 
expansion of fi^j about the point d and letting d ^ oo, with 
i < j fixed. The following statements then follow immediately. 


Corollary 1: It is only possible to maintain Aqh.z > 0 > 
AqHj when d = 0(1). From Lemma when this occurs 

lim„^oo = 00. 

We will now discuss Lemma and its corollary. These 
are seen to complement ||^ Theorem. 4], in which it was 
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shown that lim„_>.oo [limdo->.oo [cn/du] /«] (where do is the 
number of destination antennas) will be strictly positive if and 
only if d/djj = 0(n^+'^) for all e > 0 (note, the inequality 
for e is not strict). In our work, if d/j = 0(n^+'^), for 
fixed j, will be bounded away from infinity \/ i < j 

and consequently, from will almost surely display 

no exponential growth as n grows without bouncQ Clearly, 
avoiding exponential growth of required if we are 

to multiplex over the j upper eigenchannels. Crucially, these 
results provide us with an alternative perspective to Q on 
how the number of antennas (more precisely, the scaling of 
this number) at each node affects the end-to-end capacity of 
the network. 

3) Network behavior when = 0 or Aq,h.i = 0.' 
Suppose we wish to ensure that the (1, i)th normalized channel 
capacity displays no exponential growth; i.e., (from @) 
— 0- Furthermore, suppose this is achieved by ensuring 

that 

Aqh.i > Aqh.i = 0. (64) 


Then ( |46| ) and Lemma 0 give 


us 




= 0p , 


(65) 


where the argument of 0p (•) in ( |65l l is bound in the following 
way; 

< gn(Hd-i-Hd.-i) < Qd-l+1 . ( 66 ) 


Thus, ensuring = 0 implies that the transmit power must 
grow according to This growth rate is strictly positive 
and bound according to ( |6^ . We can see that by increasing 
the number of antennas, d, for a fixed i, the rate at which the 
transmit power grows can be reduced. Conversely, by fixing 
d and increasing i (i.e., multiplexing more data streams), the 
rate at which the transmit power must grow will increase. 

Suppose instead we wish to ensure that the transmit power 
displays no exponential growth by setting Aqh.i = 0. From 
( |5^ and Lemmas and this gives 

. (67) 


Thus, all of the growth properties that applied to 
when Aqh.j = 0 apply to when Aqh,i = 0. 

Remark 2: Interestingly, from ( |65l l and ( |C7] i, it can be seen 
that there is a duality between the exponential growth rate of 




(a) 


and when either A^h,* = 0 or Aqh.i = 0, 

respectively. This duality property will be exploited below. 

4) Adjacent Eigenchannel Capacity Divergence and In¬ 
dividual Data Stream Cost: For the final problem, let us 
consider the rate at which adjacent eigenchannel capacities 
diverge away from each other. Of course, we have already 
seen (Lemma and ( |5^ ) that if Aqh.j > Aqh,z-i-i > 0 then 
Ci^n and Ci+i^n will not diverge away from each other. Thus, 


^For the work in and our work, do and j can be thought of as the 
maximum number of aata streams that can be multiplexed over the channel, 
respectively. This draws the connection between that work, where the scaling 
of the ratio d/d£) is assessed, and our work, where the scaling of d/j is 
assessed. 


in what follows we consider the cases 0 > AaH.i > AaH.i+i 
and AaH.j > 0 > AaH,*+i- 

When 0 > AaH.i > AaH.j+i, by employing Lemma we 
find that 

^if+i,n = ■ ( 68 ) 

Thus, the ith and {i -f l)th channel capacities diverge away 
from each other at an exponential rate l/{d — i). When 
Aqh.i > 0 > AaH.i-i-i we find that 

= ©P = Op (e3^) (69) 

and the capacities diverge away from each other at an ex¬ 
ponential rate —2AQ,H,i+i, which is upper bounded by the 
exponential rate of ( |68] l. 

Remark 3: By considering the discussion of duality in 
Remark]^ we can assign a cost (in terms of extra instantaneous 
power requirements) to each extra data stream that we attempt 
to multiplex. In particular, from ( |68] l and because of the 
duality property, if we are multiplexing i data streams, then, 
to multiplex 1 more stream (whilst ensuring AaH.i+i = 0), we 
must increase the nth relay’s instantaneous transmit power by 
(approximately) a factor of exp{n/{d — i)). Furthermore, we 
find that the cost of each extra eigenchannel increases with i. 

VII. Numerical Illustration 

In this section, we will illustrate the theory that has been 
presented in the previous sections. It is important to mention 
firstly that the following Monte Carlo simulations were gen¬ 
erated using the variable precision arithmetic (vpa) function 
within the Matlab symbolic toolbox, which allowed us to 
increase the accuracy of our calculations to (approximately) 
100 decimal places. This is required because of the nature 
of our results: we are verifying as clearly as possible that the 
eigenchannel capacities follow exponential trends governed by 
their corresponding Lyapunov exponents. For large networks, 
this results in computational rounding within the simulations 
if vpa is not utilized. An immediate consequence of employing 
such high precision is that simulations are very computa¬ 
tionally intensive. Crucially, this restricts us to demonstrating 
network trends when the number of antennas at each node are 
small (i.e., 3 or 4). 

Figs. 1^ and illustrate the second statement of Theorem 
1^ for a 4 X 4 fixed-gain system. The first of these figures 
shows eigenchannel capacity as a function of network size, and 
clearly demonstrates that this will trend along a deterministic 
trajectory governed by the network’s Lyapunov exponents. 
The second of these figures clearly shows convergence in 
the normalized logarithm of the eigenchannel capacity to the 
network’s Lyapunov exponents. Figs. and illustrate anal¬ 
ogous results to those above, but for a variable-gain system. 
Interestingly, for all these figures we see that convergence to 
the stated trends occur quickly, sometimes in the order of 5 
to 10 hops, which attests to the utility of our methods. Figs.|^ 
and [^demonstrate ( |5^ as a function of n for a 3 x 3 variable- 
gain system. Similar plots occur for fixed-gain. As with above, 
convergence to the stated trends occurs quickly. 

Because of the issues associated with computational com¬ 
plexity (mentioned at the beginning of this section), we were 
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Fig. 2. Figure demonstrating that the capacity of each eigenchannel is given 
by = ©p for a 4 X 4 fixed-gain MIMO system, see Theorem 

Dashed lines represent exp ; solid lines represent instantaneous 

realizations of c^l-, where, starting from the top, i = I,-- - ,4. For all 
2 = 1,..., n, we set Pi = no = /ii = 1. 



Fig. 4. Figure demonstrating that the capacity of each eigenchannel is 
given by = ©p J for a 3 x 3 variable-gain MIMO system, 

see Theorem Dashed lines represent exp solid lines represent 

instantaneous realizations of where, starting from the top, i = 1, - • • ,3. 
For all 2 = 1,..., n, we set Pi = no = pi = 1. 



Fig. 3. Figure demonstrating the second statement of Theorem[^for a 4 x 4 
fixed-gain MIMO system. Dashed lines represent the Lyapunov exponents 
solid lines represent instantaneous realizations of ^ log , where, 
starting from the top, i = 1, • • ■ ,4. For alH = 1,..., n, we set Pi = no = 
Pi = 1- 


unable to employ Monte Carlo simulations to demonstrate 
(numerically) the relationship between antenna scaling with 
respect to number of hops, and the rate at which eigenchannel 
capacities diverge away from each other (see Lemma [^. We 
do, however, show Fig. which plots ^ as a function of 
the number of antennas at each node. In this figure, when 
d is large the curves are seen to decay linearly on the log- 
log scale; i.e., they decay like 0(1/d) on a linear scale. This 
observation theoretically illustrates Lemma (specifically, 
and consequently, that if super-linear antenna scaling 
occurs with respect to the number of hops within the network, 



Fig. 5. Figure demonstrating the second statement of Theorem[^for a 3 x 3 
variable-gain MIMO system. Dashed lines represent the Lyapunov exponents 
; solid lines represent instantaneous realizations of k log ], where, 
starting from the top, * = 1,2,3. For all * = 1,..., n, we set pi = no = 
Pi = 1- 


the ith and jth eigenchannel capacities will not exponentially 
diverge away from each other. 

Finally, Fig. shows an estimation of ^ and its upper 
bound ( |115| l for a variable-gain network as a function of the 
transmit power at each node for a large network size, n = 
1000. The choice of such a large n is only made to ensure 
that our results have converged significantly, where smaller 
values of n may exhibit less smooth plots. For this figure, we 
assume that the mean channel fading coefficient at the ith node 
is log-normally distributed. It is easy to see that the bound is 
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Fig. 6. Figure demonstrating that := is given by = 

0p for a 3 X 3 variable-gain MIMO system, see j56) . Dashed 

lines represent exp ^; solid lines represent instantaneous realizations 

of where, starting from the top, i = 1, ■ ■ ■ , 3. For all i = 1,..., n, 

we set Pi = TLQ = fii = 1. 



Fig. 7. Figure demonstrating j5^. Dashed lines represent the Lyapunov 
difference )50t ; solid lines represent instantaneous realizations 

of ^ where, starting from the bottom, i = 1,.. ., 4. For all i = 

1,..., n, we set Pi = no = Pi = 1. 

very tight for large PilriQ. 

VIII. Conclusion 

In this paper, we have employed the formalism of RDSs to 
study the scaling properties of the transmit power and end-to- 
end channel capacity of hnite antenna MIMO AF relay net¬ 
works. By employing the RDS formalism, we have been able 
to associate Lyapunov exponents (which are classically used 
to characterize the stability of RDSs) with the MIMO AF relay 
network. Our study has revealed that the exponential growth 
and/or decay of the transmit power and end-to-end channel 
capacity are completely characterized by the network’s Lya¬ 



Fig. 8. Figure demonstrating Lemma|^for different values of i as a function 
of the number of antennas. 



Fig. 9. Figure showing numerically estimated (curved lines) for large 

network (n = 1000) and its upper bound (straight lines), i |l 15) , for a non- 
homogeneous variable-gain 3x3 network. The average channel fading charac¬ 
teristics, fii, are assumed to be log-normally distributed with parameter pairs 
(0,1), (0, 2) and (0, 3); i.e., /x* ~ CAf{a, b), a = Elog/Xj, b = Vlog /Xj, 
. The plot is taken as a function of the normalized transmit power at each 
node, with pi = pi—i Vi. 


punov exponents. Furthermore, our methods can be applied 
to systems with arbitrary channel fading statistics, provided 
Elog+limil <C 30 , where is the channel matrix for the Ah 
hop; however, in this contribution we focus explicitly on the 
Rayleigh fading scenario. We then establish growth laws for 
the eigenchannel capacity divergence, how this relates to the 
amplihcation strategy and number of antennas at each node, 
and the cost (in terms of power) associated with multiplexing 
extra data streams. Finally, we would like to close with the 
following open question: Can our techniques be extended to 
study the capacity and power scaling properties of networks 
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employing other (non AF) forwarding strategies? 

Appendix A 
Proof of Theorem[T] 


However, if Vi H 3? = 0 is satisfied, it can be seen that because 
Vi is a vector space all vectors in Vi must have their (d+ l)th 
element equal to zero. Thus, 


Firstly, let 

n 

1]^:= [Xj 1]^, (70) 

it is easy to see from Definition 3 that A ^M, [Xq l] > 
Aa.i- Thus, from Fact[^ 

A (m, [X^ 1 ]^) e {Aa.* > Aa.i} u Aa,i =: C. (71) 

The proof of the theorem now follows from Claim (men¬ 
tioned below). 

Claim 1: With y ■= {[yi • • • : yi G C}, the mapping 

A(M,-):3^^/: (72) 

is suijective. 

Proof of Claim If Aa.i < Aa,i then C = {Aa,i}, 
A (M, Y) = Xa,i y Y G y and the surjectivity of 0 is 
satisfied. Thus, w.l.o.g., we assume that 3 k < d such that 


V, n 3^ = 0 

^ V. = {A = [yi ■■■yd 0]^ : A(M, A) < Aa.,} 

^ dimVi = dim{A'= [yi ■ ■ ■ ydf ■■ A(A, A') < AA.i} 

= d+l-i. (77) 

But from ( |75l l, dim Vi = d + 2 — i, so Vi C V = 0 gives us a 
contradiction, becomes 

(VAV,+i)n3^ = 0 3^cv,+iC---cVi, (78) 

and 

(v,\v,+i)nv = 0, Vj<z. (79) 

This proves the first part of the Claim. 

We will now prove the second part of the claim. From fTS] ) 
we have (Vi \ V 2 ) H 3^ = 0 V 2 3 3^. But y contains a d 
dimensional subspace A := {[yi ■■■ yd 0]^ : yi G C}, and 
V 2 is also d dimensional, so 

V2^y^A^A = V2^A = y. (80) 


Aa,1 > • • • > Aa.A: > Aa,l > AA,fe-|-l > ’ ’ ’ > Aa.^- (73) 

In what follows, we consider the scenario in which Aa.^ > 
Aa,i > AA.fc-i-i- The proof can easily be extended to the case 
when Aa.A; = Aq,i. 

Consider the filtration, 

{0} =: Vp+i C Vp C • • • C Vi = (74) 

where Y G Vi \ Vi+i <=> A(M, Y) = Ai (the existence of such 
a filtration is guaranteed by Fact 1^ - The proof of Claim [T] 
then follows immediately from Claim (mentioned below). 

Claim 2: Let Vi be as in ( |74| l and y be as in Claim [T] Then 
(Vi \ Vi+i) n V ^ 0 for all z = 1,..., fc + 1, where XA,k > 
Aa.l > AA,fe-|-l- 

Proof of Claim 1^ Claim 1^ follows immediately from Claim 
(mentioned below). 

Claim 3: Let Vi be as in ( |74l l, y be as in Claim[T] and suppose 
that AA.fc > Aa.i > AA.fe-i-i- Then; 

1) for all i < k, (VAV,+i)n3^ = 0 implies (VAVi+i)n3^ = 0 
for all I < i, 

2) (Vi \ V 2 ) n 3^ A 0- 

Proof of Claim We will begin by proving the first part 
of the claim. To do this, we first note the following: all the 
Lyapunov exponents have multiplicity 1 (i.e., they are distinct); 
consequently, from Fact|^ dimV^ — dimVj+i = 1 V j and 

dimVj=d + 2-j. (75) 


Clearly, 

(V. \ V,+i) n 3^ = 0 V. n 3^ = 0 or 3^ c v,+i c • • • c Vi. 

(76) 


But Vl C 3^, so from ( |80l l V 2 ^ y gives us a contradiction. 
Thus V 2 2 which (from ( |78l l) gives 

(Vi\V2)n3^ A0- (81) 


This completes the proof. 

Appendix B 


We have 
lim -log||A„ 

n—>-oo 77, 


Proof of Lemma[T] 


— lim lo§ —1 11 

n—voo Tl 

< maxj lim - log ||A„A„_i - i?„||, 

n^oo n 


lim — log||2i?„ 

n—^00 Tl 


= max<^ lim - log | |A„| |, 0 A (82) 

[^n —>^00 n J 

where the second line follows from Lemma 1^ (below) and the 
last line follows from the symmetry of and that i?„ A 0- 
If 

lim - log ||A„|| > 0, 

n—^oo Tl 

our result is reached trivially; if 

lim -log||A„|| = A < 0, 

n—^oo Tl 

from Lemma (below), the line above ( [82l ) holds with 
equality, which gives A = 0. This contradicts our assumption 
that A < 0. Therefore, 

lim - log ||Aa|| > 0. 

n—>-(x> Tl 
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This completes the proof. 

Lemma 6: For S 

lim - log ||a„ + (3n\\ 

n—xx) n 

<max| lim -log||a„||, lim -log||/?„ 

I n— >-oo Tl n—>-(x> fl 

where equality holds when 

lim -log||a„||^ lim -log||/3„||. 

n^oo 77 , n—^oo 77 

Proof: For a„,/3„ G 
lim - log ||a„ + PnW 

n—foo Tl 

< max I lim — log||a„||, lim — log||/3. 

I n—foo Tl n—>-oo Tl 


(83) 


(84) 


(85) 


k To show that 


since ||a„ + /3„|| < 2max{||a„||, ||, 

holds with equality when 

lim -log||a„|| 7 ^ lim -log||,8„||, (86) 

n—^oo Tl n—foo Tl 

w.l.o.g., we assume that lim„_>oo ^ log ||an|| < 

lim„^oo ^ log ||/3„||. Eq. ([8^ then gives us 

lim -log||a„ + /3„|| < lim -log||/3„|| (87) 

n—foo Tl n—>-oo Tl 

< lim - log ||a„ +/3„ - q;„|| 

n—>-oo Tl 

< maxi lim - log ||q!„ + |, 

[^n—)-oo Tl 

lim - log ||a„||I (88) 

71^00 Tl J 

It follows that if lim^^oo (!/'«) log | |a„ + /3n|| < 

lim„^oo ^ log 11 On 11 then (from (13 and t [88] t) 

lini^_j.Qo (l/n)logp„|| < lim„^oo(l/n)log||a„|| 

which contradicts our assumption. Consequently, 

lim„_>.oo(l/n) log ||,5„|| is sandwiched either side by 

lim„_>.oo(l/n) log ||q!„ + Pn\\ and so must be equal to it. ■ 


Appendix C 
Proof of Theorem|2] 

Theorem contains two statements. We prove these sepa¬ 
rately in the following two subsections. 

A. First Statement 

We prove the first statement in two parts. Each of these 
parts will involve manipulating the inverse of 
which is given by 

= (89) 

1=2 / 

where, without loss of generality, we have assumed that 
no = 1. The first part constructs an upper bound on the 

limit in question. The second part constructs a lower bound 
on the same limit, which is identical to the lower bound. This 
proves the first part of the theorem. 


1) Upper Bound : Our aim is to show that 
lim - log£i < mm{0, 2AaH.i}- (90) 

n—)-oo Tl \ / 

By noting that 

(rS”') 7' 0, 

and does not depend on n, we obtain 

lim - logfi “=■ 0. (91) 

Also, from the definition of Rj“^ and property of Eactj^ it 
is clear that 

lim - log£, (R[f) =■ -2AaH.*. (92) 

n—>-oo 77, \ / 

By combining ( |9T] l, ( [93 . and 

— max{0, —a} = min{0, a}, a G K, (93) 

the upper bound of ( |90l l follows immediately from the fol¬ 
lowing claim (the proof of which is given at the end of 
Appendix [^ i. 

Claim 4: The eigenvalues of are bound in 

the following way; 

> max {f, (r^-^) , £^ (rS”') } , (94) 


2) Lower Bound: We will now provide the second part 
of the proof (constructing the lower bound). To begin, let us 
introduce the following RDI^ which will be exploited in a 
moment; 


T; := M ' 


■m1“) 


1 ’ 


(95) 


where 


;= 



±1 


(96) 


To allow us to describe the mechanism by which the sign of 
±1 is chosen in the bottom right corner of ( |96l l, we must first 
establish the inner product of Y^. The inner product of y„ is 
given by 


First inner product term 


||y„f =yj R^“^ ' + y] Rg ' + Id Ik 


Second inner product term 


n - n—1 


Hr' • • • iin-i n n 


j—1 ^ j—1 


±..-± H, 


-nt 




ai a 

■ 




( 97 ) 


®Note, (951 is a backward RDS as per (241 . 
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where 

Rg-' := H-i.. - Hr' (Hr')^ • • • f[ (98) 

i=i 

It is clear that the second inner product term is a real number 
that, at the moment, may be either positive or negative. 
However, there is nothing stopping us from ensuring that this 
is strictly positive by appropriately selecting the sign of ±1 
in ( |9^ ; for, the RDS is permitted to remember the past, and 
predict the future | |28) . This is the mechanism that we will use 
to select the sign. Furthermore, performing sign selection in 
this way will not affect the Lyapunov exponents of the system 
in question (Fact [^. We now have the following upper bound 
on the first inner product term of ( |97l i, which will be exploited 
later on; 


But the limit on the right hand side of ( |105| ), when Yn is given 
by ( |95] ), is given by Theorem [T] Thus, ( |105| ) and Theorem [T] 
give us 

lim - logfi > -max{-2AaH.i,0}, 

(106) 

which can then be combined with ( |93] l to yield the lower 
bound. 


B. Second Statement 

From the first statement of Theorem we have 


log 


nYY]-o{n) _ 


+ 1 <c" 


< 


log 


_|_1 


1 , 


+ y'' + Id] Yo- 




(99) 


1=1 


It can already be seen that ( [89] l is remarkably similar to the 
first inner product term of ( |97l l. We will now show that this 
similarity is not superficial, and that 


lim 
n—xx) n 




■R (‘^) R 

^X,n^A/',n 


-1 


< lim 3 logfg ("rJI y y rS y 
V i=i / 


n—foo Tl 


( 100 ) 


To do this, note that 




Consequently, 

£. 


(( 


R (^) R 

X,n^Af,n 


= s. 


-1 


( 101 ) 


= £ R' 


(a)-l 

'I,n 


Er4“/ 


(a)-l 


1=1 


= R 






'X.n.l 




1=1 


( 102 ) 


(103) 


< £ R 




n—1 


(104) 


X,n,l 




where ( |102| ) follows from the first equality of ( |101| l, (| 103| ) 
follows from the second equality of ( |101| l, and ( |104| l follows 
trivially from ( |103| l. 

With P02| i and ( |104| i, we have shown ( |100| i. The right hand 
side of ( |99l l is known to be equal to the ith eigenvalue when 
Yq is an ith unit eigenvector. Combining this fact with ( |104| ) 
gives us 


lim — 

n—>-oo Tl 


log£ (^(rSR^^J„ ') ) - n 


( 105 ) 


which gives 

p (") + O ) 


< 


cM < (") + O ) 


e ^ 


< e"y+°(»)o(l) 




^ 1 
^ 1 
^ 1 


where the first line follows from the Taylor expansion of 
log(l+a;) about a; = 0 and the second line follows by factoring 
gnA<, >±o(n) fj-Qjjj ]-jjg jgf]- right sides of the second line, 
respectively, and noting that A^“) < 0. 

Proof of Claim An immediate consequence of the dual 
Lidskii inequality is that 

£ (A + B) > £ (A) + (B), (107) 


which applies to dxd Hermitian matrices A and B. Combin¬ 
ing ( |107| ) with the fact that the summands in ( |89l l are positive 
definite (i.e., they have positive eigenvalues), gives us 

>£:g(R^y) and (108) 

fg ((rER^^?-') > fg (h„ • •. H^Rg-' (109) 

( 110 ) 

Claim 1^ follows immediately from ( |110| i after noting that 

£ (h„ • • • HaRg-'H^-i • • • H;;i) = £ (Rg”') ■ 


Appendix D 
Proof of Lemma[2] 

Lemma contains two statements. We prove these sepa¬ 
rately in the following two subsections. 

A. First Statement 


The Lyapunov exponents of the matrix product that de- 
scribes the progression of X„, follow immediately from 


146 Proposition 1]. The Lyapunov exponents of Xn 


then follow immediately from Theorem Combining these 
with (H), we obtain ( |4^ and ( [46] ). 
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B. Second Statement 

For the second statement, we begin by showing that the 
limit is greater than or equal to zero for both fixed-gain and 
variable-gain; 


where the final line follows from Elog ||Hi||^//i^ 
From ( |4^ , the stated result follows immediately. 

References 




lim — log — > lim — log °=' (111) 

n—>00 n Pq ra —>-oo n Pq 

where the almost sure equality becomes an equality for fixed- 
gain. For fixed-gain, the stated result then follows immediately 
from ( |1 1 l| l and ( |1 14| ). For variable-gain, the stated result then 
follows immediately from ( |lll| l and ( |115| l. 

Appendix E 
Proof of Lemma[3] 

The lower bound follows trivially from (|^ and By 
noting that ^ > 0 A^“^ = A^“^ = 0, we 

obtain equality of the bound. For the upper bound, we need 
to prove that 

a — 6 > min{0, a} — min{0,6} (112) 

for a > 6. To do this, we need to check the following three 
cases: 

1) a > 0, 6 > 0, a > 6; 

2) a > 0, 6 < 0; 

3) a < 0, b < 0, a > b; 

which can be done trivially. Equality of the upper bound 
occurs when b < a < 0. Einally, to obtain the if and only 
if statements, we need to show that a > 0 and 6 < 0 implies 
that 

a — b > min{0, a} — min{0, b} > 0, (113) 

which can be done trivially. The independence of fixed-gain 
or variable-gain implementation is trivial. 


Appendix E 

Lemma 7: Eor the fixed-gain network, 

A/h.j < f lim - log — - log j + 1) 

2 \n^QO n Po 

Eor the variable-gain network. 


(114) 


A„H,i < ^ f lim - log — -f log(d) - ip {d^) 

2 \n^oo n Po 

+tlj {d - i + 1)). (115) 


Proof: The first equation, ( |114) 1, is obtained from ( |42| i by 
noting that 

L{f^fJ.)< lim —log—— logd. 
ra->oo n Po 

Eor the second equation, (|115|l, we have 


L{v^p) = lim — log 

n—foo Tl 

2=1 

1 "" 

< lim — log 

n.—Tt • ^ 


Pi 


Pi-1 


^ II Hi III- + dno 


n—fC30 n 


dpi 


k-i||h 
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01 F 


= lim — log — + log(d) —(116) 

n—)-oo n Po 
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